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Q so that the background image matches the theme, inserting the background image as a background in a second display area and 
^ superimposing the text communications on top of the background image. The graphical chat user interface is can display avatar and 
^ action images as well. 



WO 03/045058 Al I Hill 1DII1II II IIIHI IIIII 1)11 1 11 1)1 lllll IHII Hill Illl IIIIID IIII IHI IIII 



Published: For two-letter codes and other abbreviations, refer to the "Guid- 

— with international search report ance Notes on Codes and Abbreviations " appearing at the begin- 

ning of each regular issue of the PCT Gazette. 



WO 03/045058 



PCT/EP01/14281 



1 

TELEVISION CHAT ROOM 

The present invention relates generally to an entertainment system, and more 
specifically to concurrent viewing of video programming and of text 
5 communications between viewers of the video programming. Still, more 
particularly, the invention concerns a graphical chat user interface. 

A chat room allows users to communicate on-line with other computer or 
set-top box users, typically over the Internet. Chat rooms are often named 

10 after discussion topics in order to facilitate for users to enter a chat room 
where the discussion pertains to their desired topic. Client software such as 
e.g. Internet Relay Chat (IRC) or America Online 4.0 features a text window 
that displays each message submitted by users visiting the room. These user 
interfaces provide controls for entering a message and submitting entered 

1 5 messages to the server system. 

Early on it was necessary to have a computer and a television set in the same 
room in order to chat about a certain television show and watch the show 
simultaneously. Later developments made it possible to watch television on a 

20 computer equipped with a television card or to chat on a television set with the 
help of an Internet enabled set-top box. An Internet enabled set-top box 
essentially has the same components as a general purpose computer except 
that it also includes a television tuner for receiving broadcast (satellite or 
terrestrial) and/or cable television signals. 

25 While graphical chat user interfaces for computer chat rooms have developed 
into virtual three dimensional worlds where users can move around television 
chat rooms have retained a less advanced structure consisting mainly of 
dividing the display into one television area and one chat area. 



30 



U.S. Patent No. 5,694,163 discloses a computer program for viewing of chat 
over associated data provided by way of on-line computer services in 
conjunction with a broadcast television program. A chat is formatted to 
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provide a television compatible chat TV signal, which is inserted into a 
television program TV signal and broadcast. The encoded TV signal is 
received by a computer and separated into an audio/visual portion and an 
encoded data portion. Viewers who do not desire to participate in the chat, or 
5 do not have the capability of participating in the chat, are allowed to view the 
conversations of the chat at the time of viewing the related television program. 
Another aspect of the invention provides a telephone back channel at a user's 
computer that allows connection to an on-line service provider via the 
computer's modem. 

10 

U.S. Patent No. 5,880,731 discloses a method for communicating a gesture 
by an avatar that represents a participant in an on-line graphic chat session. 
Each participant in the chat session is enabled to select one of a plurality of 
different avatars to represent the participant in a graphic chat session. 
15 Associated with each avatar is a bitmap file that includes a plurality of frames 
illustrating the avatar in different poses, actions, and emotional states. 
Selected frames are displayed in rapid sequence in accord with a script file to 
create an animation effecting each gesture. 

20 PCT Patent Publication No. WO 00/64150 discloses a method for linking chat 
rooms and television viewing. A video signal, such as a television program, is 
received and displayed to a viewer. The system has a user-selectable chat 
option. A transcript of the text messages from all of the viewers who have 
selected the chat option is displayed to all such viewers. The video signal is 

25 simultaneously displayed on a graphical user interface having a video region 
for display of the video signal and a chat region for display of the transcript of 
text messages. 

PCT Patent Publication No. WO 01/50753 discloses a method of assigning a 
30 user to a network chat room. The method includes determining television 

programming viewed by a network user at a networked device and assigning 
the network user to a chat room corresponding to the determined television 
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programming. The television programming identification may be a channel 
number, a television network identifier, and/or an identifier of a particular 
television program. 

5 PCT Patent Publication No. WO 01/50753 further discloses a TV-enhanced 
buddy list. The user can define a list of server system user names. In addition 
to showing whether users on the list are currently logged on to the server, the 
enhanced buddy list shows the TV show or network each listed buddy is 
watching. By selecting a link, a user can watch the same show that a friend is 
10 watching. 

The object of the present invention is to provide a method, a system and a 
computer program product for concurrent viewing of video programming and 
of text communications between viewers of video programming. Various 
1 5 implementations of the invention may include one or more of the following 
features. 

A method for providing a graphical chat user interface on a display associated 
with a multimedia apparatus, comprising the steps of: receiving a broadcast 

20 video signal; receiving text communications from at least one other television 
viewer; displaying said broadcast video signal in a first display area on said 
display; determining the identity of the television programming displayed in 
said first display area; determining the theme of said television programming 
identity; selecting a background image from an image database, said 

25 background image matching said theme; inserting said background image as 
a background in a second display area on said display; and superimposing 
said text communications on top of said background image in said second 
display area. 

30 A multimedia apparatus comprising a first receiving means for receiving a 
broadcast video signal, a second receiving means for receiving text 
communications originating from a plurality of multimedia apparatuses, and 
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for receiving a plurality of images, a memory means for storing said plurality of 
images, a determination means for determining the identity of the television 
programming tuned to by said first receiving means, and for determining the 
theme of said television programming identity, a display means for displaying 
5 said broadcast video signal in a first display area, for displaying a background 
image, retrieved from said memory means and matching said theme, in a 
second display area, and for displaying said text communications 
superimposed on said background image in said second display area. 

10 A computer program product, disposed on a computer readable storage 
medium, comprising computer readable program code means for causing a 
computer to perform the method of claim 1 . 

Other objects and features of the present invention will become apparent from 
15 the following detailed description considered in conjunction with the 

accompanying drawings. It is to be understood, however, that the drawings 
are designed solely for purposes of illustration and not as definition of the 
limits of the invention, for which reference should be made to the appended 
claims. It should be further understood that the drawings are not necessarily 
20 drawn to scale and that, unless otherwise indicated, they are merely intended 
to conceptually illustrate the structures and procedures described herein. 

In the drawings, wherein like reference characters denote similar elements 

throughout the several views: 
25 Figure 1 is a schematic view of a multimedia system for use in 

accordance with the present invention. 

Figure 2 is a schematic block diagram of the multimedia 

apparatus shown in Figure 1. 

Figure 3 is a schematic view of a user interface according to the 
30 present invention, illustrating a text communications superimposed on a 

background image that matches the theme of the displayed television 

program. 
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Figure 4 is a schematic view of a user interface according to the 
present invention, illustrating avatar images, text communications and actions 
images. 

Figure 5 is a schematic view of a user interface according to the 
5 present invention, illustrating a main section and a VIP section. 

Figure 1 depicts a multimedia apparatus 100 which is part of a client system. 
The multimedia apparatus 100 can be a set-top box, a computer or an 
electronics unit built into a television set. The multimedia apparatus 100 

10 receives user input from either a remote control 102a, a keyboard 102b, a 
mouse 102c or any combination thereof. A display 104, such as a television 
set or a computer monitor is connected to the multimedia apparatus 100, via a 
communications link 106, and used for displaying video data and a graphical 
user interface as well as emitting and audio data. The multimedia apparatus 

15 100 includes hardware and/or software for receiving and decoding a 

broadcast video signal 108. The broadcast video signal 108 may be broadcast 
via satellite, cable or terrestrial means. The multimedia apparatus 100 may 
include a modem and/or other equipment such that a bi-directional 
communications link 110 can be established for telephone, ISDN or Ethernet 

20 connections. 

The user controls television viewing and other features of the multimedia 
apparatus 100 with the remote control 102a. The remote control 102a typically 
includes a keyboard but there may also be a stand-alone keyboard 102b. A 
25 user can enter text data through either of these input means. Optionally, a 

mouse 102c could be used in combination with a virtual keyboard represented 
on the display 104. The multimedia apparatus receives power through a 
power line (not shown). 

30 The multimedia apparatus 100 is shown in more detail in Figure 2. The 
multimedia apparatus 100 includes a central processor unit (CPU) 200, an 
input output (I/O) unit 202, a memory 204 that stores data and various 
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programs such as an operating system 206 and one or more application 
programs 208, and a hard disk drive 210 that provides non-volatile storage of 
computer-readable instructions, data structures, such as an image database 
212, program modules and other data. 

5 

The multimedia apparatus 100 further includes a transport stream parser 214 
for receiving satellite, cable or terrestrial transmissions in digital MPEG-2 
format from e.g. a dish antenna 216. The transport stream parser 214 
recovers signal formatting information. A selected channel from the real-time 
0 transmission is fed to a conventional video decoder 218, which converts 
digital signals into a suitable RF modulated form to be supplied to an aerial 
socket of a conventional television set. These components are interconnected 
by common bus 220 in a manner well known per se. The bus 220 may be any 
of several types of bus structures such as a memory bus, a peripheral bus or 
a local bus. 

Access to the Internet is provided through the input output (I/O) unit 202 and 
the bi-directional communications link 110, e.g. via the telephone network 
222. The display 104 is also connected to the bus 220 via a interface 224 
such as a video adapter. 

The multimedia apparatus 100 provides a television chat functionality to the 
user by activating an application program 208 stored in the memory 204. This 
functionality is accomplished by receiving a broadcast video signal 108 and 
displaying the television program on the display 104 together with text 
communications, hereafter referred to as chat communications, received from 
other television viewers either via the bi-directional communications link 110 
or together with the broadcast video signal 108. In the latter case the chat 
communications are multiplexed into the broadcast stream. Further, the user 
is able to send own text comments to the chat service provider via the input 
output (I/O) unit 202 and e.g. the telephone network 222. The television 
program may be shown in a diminished size or part of the available television 
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image may not be shown at all. In Figure 3 the television program is shown in 
a television display area 300 occupying the top half of the display 104 and the 
chat communications is shown in a chat display area 302a. 

5 Figure 3 further depicts a chat background 302b, consisting of five cactuses 
304 and one wagon 306, that coincides with the chat display area 302a. The 
chat communications 308 is superimposed on the chat background 302b. The 
background image can have a size which is adapted to a certain setting of the 
television display area 300 and the chat display area 302a, such as full length 
10 and half the height. However, the background image is preferably a full length 
and a full height sized image so that it can be used in any type of setting. In 
Figure 4 the use of a full size image is shown where a part of the image has 
been removed to fit the setting, and as a result all items, e.g. cactus 400, are 
not completely visible. 

15 

In the preferred embodiment, the image database 212 is downloaded to the 
multimedia apparatus 100 and stored in the hard disk drive 210 prior to 
employing the invention. During chat a suitable image is selected and 
displayed as a chat background 302b in the chat display area 302a. In 
20 another embodiment the image database 212 is residing on the Internet and 
suitable background images are downloaded when needed and temporarily 
stored in the memory 204 or in the hard disk drive 210. 

In yet another embodiment, when the chat communications is multiplexed into 
25 the broadcast stream and received together with the broadcast video signal 
108, the chat service provider will send suitable background images from the 
chat server in the broadcast stream. 

The chat functionality is further enhanced by matching the chat background 
30 302b to the theme of the television program shown in the television display 
area 300. In the preferred embodiment this is achieved in the following way; 
the channel watched by the user is determined in a manner well known per 
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se, an electronic program guide inherent in the multimedia apparatus 100 is 
consulted to retrieve the identity of the television programming, available 
guide data about the television program is used to determine its theme and 
the background image that best matches the theme is selected to be 
5 displayed in the chat display area 302a. In another embodiment the channel 
watched by the user is determined in a manner well known per se and 
transmitted over the input output (I/O) unit 202 and the bi-directional 
communications link 1 10 to the chat server where the identity and theme of 
the television programming are determined so that an instruction, for selecting 
10 the most appropriate background image, can be returned. 

The chat background 302b may be replaced one or more times during the 
duration of a single television program. In the preferred embodiment the 
multimedia apparatus 100 will receive a trigger to replace the present 

15 background image with a new one. The trigger is provided by the chat server 
together with the chat communications 308. In another embodiment there is a 
timer function inherent in the multimedia apparatus 100 that automatically 
changes the chat background 302b after a predetermined amount of time has 
elapsed. In yet another embodiment the user can change the chat 

20 background 302b himself or herself whenever desirable. 

Figure 4 depicts four avatar images 402 displayed in the chat display area 
302a superimposed on the chat background 302b. An avatar is a graphical 
representation of a physical person in a virtual environment. Avatar images 

25 402 are stored in the image database 212 together with the background 
images. Each avatar image 402 displayed in the chat display area 302a 
represents one chat participant. The same avatar graphic can be used for all 
chat participants but preferably there will be a number of avatar graphics to 
choose between. The user will choose an avatar image 402 from a gallery 

30 and this selection is forwarded to the chat server together with the chat 

communications of the user. The avatar selection is then distributed to other 
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chat participants together with the chat communications in order for the user 
to be correctly represented. 

There may be name tags 404 displayed close to the avatar images 402 in 
5 order to identify the chat participants. Instead of displaying the chat 

communications 308 on consecutive lines in the chat display area 302a as 
shown in Figure 3, the text communications may, according to the invention, 
be displayed close to the avatar image 402 representing the chat participant 
who made the comment. Figure 4 depicts the preferred embodiment when a 
10 chat communication 308 is displayed in a callout box 406. Since there is no 
longer any need to scroll the messages in the chat display area 302a, a chat 
communication 308 can remain visible in the callout box 406 until a new chat 
communication 308 from the same chat participant. 

15 The invention makes it possible for chat participants to perform actions. All 
possible actions are predefined in the application program 208 running the 
chat functionality on the multimedia apparatus 100. Action images 408, 410 
are stored in the image database 212. In practice, an action image 408, 410 
can be several images that replace each other in order to give the effect of 

20 animation, such as an animated picture similar to those found on the Internet, 
or it can be one single image that moves across the display. Once a user 
performs an action, the selected action is forwarded to the chat server 
together with the chat communications of the user. The action selection is 
then distributed to other chat participants together with the chat 

25 communications in order for the action to be correctly represented. Normally, 
action images 408 are portrayed within the boundaries of the chat display 
area 302a. However, the invention can also be implemented to allow action 
images 410 to be portrayed in the television display area 300. In the latter 
case it may be necessary to make the action image transparent so that it does 

30 not block television viewing entirely in the area where it is portrayed. 
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Finally, according to another aspect of the invention, a record of favourite chat 
friends can be created by the user of the multimedia apparatus 100. The 
record will be stored in the memory 204 or in the hard disk drive 210. Chat 
communications 308 received from chat participants included in the record of 
5 favourite chat friends will always be displayed. Optionally, there may be a VIP 
(very important person) section 500, as shown in Figure 5, in the chat display 
area 302a where avatar images 402 of chat friends are displayed. It deserves 
to be pointed out that the multimedia apparatus 1 00 may receive chat 
communications 308 from more chat participants than are superimposed on 

10 the chat background 302b. When the application program 208 providing the 
chat functionality receives a chat communication 308, it will determine if the 
identity of the chat participant who sent the chat communication 308 matches 
the identity of any of the avatar images 402 displayed. If there is a match then 
the chat communication 308 will be displayed, but otherwise it will be 

15 discarded. Some of the avatar images 402 displayed will not be included in 
the record of favourite chat friends. Optionally, there may be a main section 
502, as shown in Figure 5, in which such avatar images 402 are displayed. 

The chat friends included in the favourite record do not need to watch the 
20 same television program as the user of the multimedia apparatus 100 in order 
for their chat communications 308 to be displayed. The advantage is that the 
user does not have to bother about finding out in which chat room his or her 
friends are located and the user can switch between television programs 
without loosing contact with them. Further, the chat friend may recommend 
25 the user to switch to whatever program the chat friend is watching. If, 

however, the chat friends are not online, their avatar images 402 may be 
depicted in black and white only, or in another suitable way, to indicate that 
they are not chatting at the moment. All other avatar images 402, e.g. those in 
a main section 502, preferably belong to active chat participants that watch 
30 the same television program as is being displayed in the television display 

area 300. However, the record of favourite chat friends is an individual record. 
Even if "Michael" is a chat favourite of "Susan's" it does not make "Susan" a 
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chat favourite of "Michael's". The record can be continuously updated by the 
user. 

There can also exist a change option. If the user of the multimedia apparatus 
5 100 becomes bored with one or more of the chat participants he or she may 
replace one or more of them. By triggering the change option the application 
program 208 will remove any given number of avatar images 402 and replace 
them with new avatar images 402 representing other chat participants. For 
example, if a chat room has 1 00 participants and the chat interface can only 

10 display 20 participants in the main section 502, then there is clearly room for 
replacing the avatar images 402 a couple of times in order to find new friends. 
Preferably, the user should be able to select which avatar images 402 that will 
be replaced before triggering the change option. It is obvious from what is 
said above that a message from one participant in a chat room is not 

1 5 automatically echoed to all of the participants in the same chat room. 

While the invention has been particularly shown and described with reference 
to preferred embodiments thereof, it will be understood by those skilled in the 
art that the foregoing and other changes in form and detail may be made 
20 therein without departing from the scope of the invention. 
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CLAIMS 



1 . A method for providing a graphical chat user interface on a 
display associated with a multimedia apparatus, comprising the steps of: 

5 - receiving a broadcast video signal; 

- receiving text communications from at least one other 
television viewer; 

- displaying said broadcast video signal in a first display area on 

said display; 

10 - determining the identity of the television programming 

displayed in said first display area; 

- determining the theme of said television programming identity; 

- selecting a background image from an image database, said 
background image matching said theme; 

15 - inserting said background image as a background in a second 

display area on said display; and 

- superimposing said text communications on top of said 
background image in said second display area. 

2. The method of claim 1 , further comprising the steps of: 

20 - selecting at least one avatar image from said image database, 

said at least one avatar image being representative of one of said other 
television viewers; and 

- displaying said at least one avatar image superimposed on top 
of said background image in said second display area. 
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3. The method of claim 1 or 2, wherein the steps of selecting a 
background image from an image database, said background image matching 
said theme and inserting said background image as a background in a second 
display area on said display are performed at least two times during the 

5 broadcast of said television programming. 

4. The method of claim 1 or 2, wherein said image database is 
residing in a memory device associated with said multimedia apparatus. 

5. The method of claim 1 or 2, wherein said image database is 
residing on the Internet. 

10 6. The method of claim 1 or 2, further comprising the step of: 

- downloading said image database via the multiplexed 
broadcast stream containing said broadcast video signal. 

7. The method of claim 2, wherein text communications received 
from said one of said other television viewers is superimposed on top of said 

15 background image in said second display area close to the displayed said at 
least one avatar image. 

8. The method of claim 7, wherein said text communications 
received from said one of said other television viewers remains superimposed 
on top of said background image in said second display area close to the 

20 displayed said at least one avatar image until another text communications is 
received from said one of said other television viewers. 

9. The method of claim 2, further comprising the steps of: 
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- receiving an action input from at least one television viewer 
associated with one avatar image currently displayed in said second display 
area; 

- selecting at least one action image from said image database, 
5 said at least one action image corresponding to said action input; and 

- displaying said at least one action image superimposed on top 
of said background image in said second display area. 

10. The method of claim 2, further comprising the steps of: 

- receiving an action input from at least one television viewer 
10 associated with one avatar image currently displayed in said second display 

area; 

- selecting at least one action image from said image database, 
said at least one action image corresponding to said action input; and 

- displaying said at least one action image superimposed on top 
15 of said television programming in said first display area. 

1 1 . The method of claim 10, wherein the at least on action 
image is transparently superimposed. 

12. The method of claim 2, further comprising the steps of: 

- creating at least one record associated with said multimedia 
20 apparatus, said at least one record comprising a list of preferred other 

television viewers; and 

- superimposing text communications and displaying at least 
one avatar image associated with at least one other television viewer included 
in said record, and superimposing text communications and displaying at least 
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one avatar image associated with at least one other television viewer not 
included in said record. 

13. The method of claim 12, wherein said at least one other 
television viewer not included in said record is watching television 

5 programming determined to have the same television programming identity as 
the television programming displayed in said first display area. 

14. The method of claim 12, further comprising the steps of: 

- receiving a change input from the television viewer associated 
with said multimedia apparatus; 
10 - replacing the avatar image of at least one other television 

viewer with an avatar image of at least one other television viewer based on 
said change input. 

15. A multimedia apparatus comprising a first receiving means 
for receiving a broadcast video signal, a second receiving means for receiving 

15 text communications originating from a plurality of multimedia apparatuses, 
and for receiving a plurality of images, a memory means for storing said 
plurality of images, a determination means for determining the identity of the 
television programming tuned to by said first receiving means, and for 
determining the theme of said television programming identity, a display 

20 means for displaying said broadcast video signal in a first display area, for 
displaying a background image, retrieved from said memory means and 
matching said theme, in a second display area, and for displaying said text 
communications superimposed on said background image in said second 
display area. 
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16. The multimedia apparatus of claim 15, wherein said display 
means also are arranged for displaying a plurality of avatar images, retrieved 
from said memory means and each avatar image corresponding to one of 
said plurality of multimedia apparatuses, superimposed on said background 

5 image in said second display area. 

17. The multimedia apparatus of claim 16, wherein said second 
receiving means also are arranged for receiving an action input from one of 
said plurality of multimedia apparatuses associated with an avatar image 
currently displayed in said second display area, and said display means also 

10 are arranged for displaying an action image, retrieved from said memory 

means and corresponding to the character of said action input, superimposed 
on said background image in said second display area. 

18. A computer program product, disposed on a computer 
readable storage medium, comprising computer readable program code 

15 means for causing a computer to perform the method of claim 1 . 

19. A computer program product, directly loadable into the 
internal memory of a digital computer, comprising software code portions for 
performing the method according to claim 1 when said product is run on a 
computer. 
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